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1 . Reference is made to the following document: 

D1: US-A-5 193 001 (Kerdranvrat Michel) 9 March 1993 

2. Item V: Reasoned statement under Rule 66.2(a)(ii) with regard to novelty, 
inventive step or industrial applicability; citations and explanations 
supporting such statement 

The present application meets the requirements of Article 33(2) PCT because the 
subject matter of claim 1 is novel and involves an inventive step in the sense of 
Article 33(3) PCT, the reasons being as follows: 

As to claim 1 : 
D1 discloses: 

- Method of movement estimation for a sequence of images (see column 2, lines 
30-34) including 

- segmentation of the video image into image blocks (see column 2, lines 36-37), 

- movement estimation per image block in order to obtain a movement vector field 
for said current image (see column 2, lines 37-39), 

- a stage of reassignment of a vector to a block by selecting one movement vector 
from among N predominant vectors (see column 2, lines 40-48), characterized in 
that 

- the predominant vectors are the ones of a group of vectors belonging to the 
movement vector field of said current image and at least to the movement vector 
field of a preceding image (see column 2, lines 40-48 supported by column 2, 
lines 30-34 and column 3, lines 31-39), 

D1 , however, does not disclose: 

- the vectors being scaled according to the temporal distance to which they 
correspond. 

This last feature is not disclosed in any of the available prior art. An inventive step 
(Article 33(3) PCT) can be acknowledged. 
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3. Item VII: Certain defects in the international application 

Although claim 1 is drafted in the two-part form the features "the predominant 
vectors are the ones of a group of vectors belonging to the movement vector field 
of said current image and at least to the movement vector field of a preceding 
image" is incorrectly placed in the characterising portion, as it is disclosed in 
document D1 in combination with the features placed in the preamble (Rule 6.3(b) 
PCT). 

The features of the claims are not provided with reference signs placed in 
parentheses (Rule 6.2(b) PCT). 

Contrary to the requirements of Rule 5.1(a)(ii) PCT, the relevant background art 
disclosed in the document D1 is not mentioned in the description, nor is this 
documents identified therein. 

The description is not in conformity with the claims as required by Rule 5.1(a)(iii) 
PCT. Care should be taken during revision, especially of the introductory portion 
including any statements of problem or advantages, not to add subject-matter 
which extends beyond the content of the application as originally filed (Article 
34(2)(b) PCT). 
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1. Method of movement estimation including segmen- 

tation of the video image into image blocks, movement 
5 estimation per image block in order to obtain a move- 
ment vector field, characterized in that it includes a 
stage of reassignment of a vector to a block by select- 
ing one movement vector from among N predominant vec- 
tors belonging to the vector field. 
10 2. Method according to Claim 1, characterized in 

that, for a predominant vector/ second-order regional 
maxima are detected so as not to be taken into account 
during the selection of the other predominant vectors. 

3. Method according to Claim 1, characterized in 
15 that the predominant vectors are selected in each of 

the four directions. 

4 . Method according to Claim 1, characterized in 
that the selection of the reassigned vector is based on 
the value of the inter-displaced- image difference 

20 (DFD) . 

5. Method according to Claim 4, characterized in 
that, if the DFDs associated with the N predominant 
vectors are greater than the DFD associated with the 
original vector, the zero vector is adopted. 

25 6. Method according to Claim 4, characterized in 

that, if the DFDs associated with the N predominant 
vectors are greater than the weighted DFD associated 
with the original vector, the original vector is kept. 

7. Method according to Claim 1, characterized in 
30 that the selection of the reassigned vector is based on 

the calculation of the activity (spatial gradient) in 
the inter-image difference block (current block - 
estimated block) . 

8. Method according to Claim 7, characterized in 
35 that, if the activities corresponding to the N predomi- 
nant vectors are greater than the activity correspond- 
ing to the original vector, the zero vector is adopted. 

9. Method according to Claim 7, characterized in 
that, if the activities corresponding to the N predomi- 
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nant vectors are greater than the weighted activity 
corresponding to the original vector, the original vec- 
tor is kept. 

10. Method according to Claim 4, characterized in 
that the components of the vectors used during the DFD 
calculations are the spatially filtered components. 

11. Method according to Claim 7, characterized in 
that the components of the vectors used during the spa- 
tial-gradient calculations are the spatially filtered 
components . 

12. Method according to Claim 1, characterized in 
that, for each image, the predominant vectors are cho- 
sen from among the field of vectors of the current 
image and the field of vectors of at least one preced- 
ing image . 

13. Method according to Claim 12, characterized in 
that the vectors of the preceding images, in addition 
to being scaled, are weighted as a function of the tem- 
poral distance. 

14. Method according to Claim 12, characterized in 
that, when a break in movement is detected, the vectors 
of the preceding images are not considered. 
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(57) Abstract 



The method includes segmentation of the video image into image blocks, movement estimation per image block in order to obtain a 
field of movement vectors. It is characterized in that it includes a stage of reassignment of a vector to a block by selecting one movement 
vector from among N predominant vectors belonging to the field of vectors. The applications relate to movement estimation, for example, 
by image-block matching. 
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METHOD OF MOTION ESTIMATION FOR TRANSMISSION 
COST REDUCTION OF MOTION VECTORS 



The invention relates to a method of movement 
5 estimation applied to MPEG-type video coding. 

The majority of movement-estimation algorithms 
implemented in video coding use the technique of "block 
matching" . 

The image is segmented into blocks of size N*N, 
10 called macroblocks, and the estimator searches for the 
vector minimizing the difference between a block of the 
current image and a block of the reference image. This 
difference is generally an MSE (Mean Square Difference) 
or MAE (Mean Absolute Difference) calculated on the 
15 luminance pixels. 

This type of estimator can supply a heteroge- 
neous movement field since it is based on the varia- 
tions of luminance and not on the actual movement in 
the sequence. This may entail an overhead for the cod- 
20 ing of the vectors by the coder, the coding generally 
being of differential type, and thus a reduction in 
performance . 

The object of the invention is to remedy the 
abovementioned drawbacks. 

25 Its subject is a method of movement estimation 

including segmentation of the video image into image 
blocks, movement estimation per image block in order to 
obtain a movement vector field, characterized in that 
it includes a stage of reassignment of a vector to a 

30 block by selecting one movement vector from among N 
predominant vectors belonging to the vector field. 

According to one particular implementation, for 
a predominant vector, second-order regional maxima are 
detected so as not to be taken into account during the 

35- selection of the other predominant vectors. 

According to another implementation, the pre- 
dominant vectors are selected in each of the four 
directions . 
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According to a particular implementation of the 
method, the selection of the reassigned vector is based 
on the value of the inter-displaced-image difference 
(DFD) . 

5 A particular characteristic of the invention 

consists in adopting the zero vector if the DFDs asso- 
ciated with the N predominant vectors are greater than 
the DFD associated with the original vector, or in 
actually keeping the original vector if the DFDs asso- 

10 ciated with the N predominant vectors are greater than 
the weighted DFD associated with the original vector. 

According to another implementation of the 
method, the selection of the reassigned vector is based 
on the calculation of the activity (spatial gradient) 

15 in the inter-image difference block (current block - 
estimated block) . If the activities corresponding to 
the N predominant vectors are greater than the activity 
corresponding to the original vector, the zero vector 
is adopted. If the activities corresponding to the N 

20 predominant vectors are greater than the weighted 
activity corresponding to the original vector, the 
original vector is kept. 

According to another particular implementation 
of the method, for each image, the predominant vectors 

25 are chosen from among the field of vectors of the cur- 
rent image and the field of vectors of at least one 
preceding image . 

By virtue of the invention, the movement vector 
fields calculated by an estimator of the "block match- 

30 ing" type can be homogenized. 

The characteristics and advantages of the 
invention will emerge better from the following de- 
scription, given by way of example and by reference to 
the attached figures, in which: 

35 - Figure 1 represents a histogram of the move- 

ment vectors, 

- Figure 2 represents a regional-maxima search 

window, 
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- Figure 3 represents an example of median fil- 
tering, 

- Figure 4 represents an example of the preced- 
ing image vectors being taken into account, 

5 - Figure 5 represents movement-vector fields 

during a zoom, 

- Figure 6 represents various types of movement 
which may be detected. 

The homogenization of the vector field is 
10 obtained via a method of conditional reassignment. 

The vectors, associated with the images of a 
sequence, are calculated and stored by the estimator. 

In order to carry out processing on the vec- 
tors, a two-dimensional histogram is constructed with 
15 dimensions of 512*512 in which the coordinates repre- 
sent the values (dx, dy) which are the values of the 
horizontal and vertical components of these vectors. 

Figure 1 represents, on the left-hand part, an 
image consisting of macroblocks to which the movement 
20 vectors are allocated and, on the right-hand part, the 
corresponding histogram. 
Choice of predominant vectors 

In order to make the movement field more homo- 
geneous, the idea is to adopt a certain number of vec- 
25 tors, which is fixed in the first place by the user. 
This number will be larger in proportion to the hetero- 
geneity of the movements. 

The first solution consists in adopting the N 
vectors corresponding to the highest frequencies of 
30 appearance. 

Another possibility is to stipulate that the 
algorithm choose N/4 predominant vectors in each of the 
four orientation planes. This solution can be adopted 
as an option, as an output criterion upon detection of 
35 zoom in the sequence.. This is because such a phenomenon 
entails distribution in all directions of the vector 
field. 

The last solution envisaged is to carry out 
detection of the regional maxima. This is because the 
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problem, in the first solution, is that it is possible 
to have several contiguous maxima, which do not confer 
enormous advantages compared with the fact of adopting 
fewer of them. 

5 The histogram is therefore scanned, rejecting 

those vectors among the N predominant vectors appearing 
in the vicinity of other more predominant vectors. Thus 
the existence of these second-order maxima is identi- 
fied by looking at the histogram to see whether two 
10 maxima lie in the same window, for example with dimen- 
sions 3*3. 

Figure 2 represents such a window, referenced 
1, for searching for regional maxima, this window being 
centred around the predominant vector adopted (dX, dY) , 

15 the number of occurrences of which is n. 

Choice of the vector allocated to a macroblock MB. Re- 
assignment 
- Method of the DFD 

Once the predominant vectors have been 

20 extracted, a criterion remains to be found for reas- 
signing each of these vectors to each MB. Since the 
movement estimator uses the criterion of the minimum 
DFD ( Displaced-Frame Difference) to calculate the move- 
ment vectors, it seems useful to use this criterion to 

25 find the best possible correspondence between the vec- 
tors adopted and the macroblocks of the image to be 
processed. 

After ordering the vectors in increasing order 
of their frequency of appearance, the calculation of 
30 DFD associated with each of these vectors is carried 
out for each MB. This calculation can be expressed sim- 
ply by the following formula: 

N-l N-l 

Dfd(i,j)= £ £ | MBCurrent (i+k,j+l) -MBReference (±+k+dy, j + l+dx) \ 

k = 0 1 = 0 



35 in which (i, j) are the coordinates of the 

MB to be processed; 

N (= 16) is the size of the MB; 
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(dx, dy) are the components of the 
vector to be tested, belonging to [-128; +127.5]. 

It is important, before applying this formula, 
to check that the vector to be tested does not point 
5 outside the reference image. If no vector is suitable, 
then the zero vector is assigned. 

Hence the vector corresponding to the minimum 
DFD is assigned to each MB. 

- Gradient method 
10 This consists in seeking, for each MB of the 

"difference" image consisting of the predicted refer- 
ence image and of the current image, the vector corre- 
sponding to the minimum gradient which gives informa- 
tion on the local activity of the MB (of horizontal and 
15 vertical gradient type) . 

MB_gradient= ^ block _ active 

41uma 
blocks 



20 



with: 



block active=MAX 



^i = 6, j = 7 

MAX 

i, j = 0 



x(i, j) - x(i + !, j) 



i = 7,j = 6 
MAX x(i, j) - x(i, j + 1) 
i,j = 0 ' 1 



Enhancement, of the reassignment 
DFD/Gradient criterion 

In order to keep certain movements, relating to 
25 objects of small size, the following criterion is 
defined: 

If, after application of the DFD method, the 
vector adopted for an MB generates a DFD greater than 
the weighted original DFD, the original vector is kept. 

30 Likewise, regarding the method of the gradient, 

for each MB obtained after inter-image difference, the 
gradient obtained by reassignment is compared with the 
gradient of the original vector. If the weighted origi- 
nal gradient is less than the new gradient, the origi- 

35 nal vector is kept. 
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Filtering applied to the movement vectors 

In order to make the vector fields more homoge- 
neous, other criteria may be used, namely spatial or 
temporal filtering. 
5 - Spatial filtering 

The filter adopted is the two-dimensional 3*3 
median filter: 

the principle is explained below in the light 
of Figure 3 which represents an image referenced 2 
10 before filtering and an image referenced 3 after fil- 
tering. The vector referenced 4 is the vector- to be 
processed. 

The vertical and horizontal neighbours of the 
components of the MB in question are ordered along each 

15 direction (dx, dy) , then the median value of each com- 
ponent is taken. Next the various DFDs associated with 
each MB are compared, in the case in which either one 
component is filtered, or both, or no component is fil- 
tered. Hence the vector corresponding to the minimum 

20 DFD is chosen, the original DFD, obviously, being 
weighted. 

- Temporal filtering 

The idea of temporal coherence is to take 
account, in the reassignment of the vectors of an im- 
25 age, of the movement fields of the preceding images; 
this is done with a view to limiting the disparity in 
the movements from one image to another. 

To begin with, we will detail the principle of 
temporal filtering of Forward vectors (deferred- 
30 movement vectors) . 

Spatio-temporal histogram of Forward vectors: 

In order to take account of the various histo- 
grams, scaling of the vectors is carried out at a first 
stage, then weighting of the occurrences which is a 
35 function of the position of the various histograms with 
respect to the histogram processed. 

Hence, for the P image of Figure 4, it is pos- 
sible to add to the histogram of original vectors, the 
occurrences of which have been weighted by a factor 3, 
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the occurrences of the vectors of the first B (the 
amplitude of which has been multiplied by 3) which are 
weighted by a factor 1 as well as the occurrences of 
the vectors of the second B (the amplitude of which has 
5 been multiplied by 3/2) which are weighted by a factor 
2. 

Temporal coherence should be relevant when uni- 
form movements are present, and breaks in movement 
(change of scene) are not present. 
10 Case of Backward vectors (anticipated-movement 

vectors) ■ 

It would be logical to think that, if there are 
uniform "Forward" movements from one image to the next, 
they would also be present in the case of the "Back- 

15 ward" vectors associated with the B images. In order to 
filter the latter, it must not be forgotten that the 
Backward vectors are based on the P or the I which will 
follow the B in question. Hence, for the first B, it 
may be thought that its Backward vectors will be twice 

20 as large as the Backward vectors associated with the 
second B. Scaling is carried out on the vectors of the 
latter by a factor of 2, and the weighted occurrences 
will be added, in the histogram associated with the 
first B. 

25 Detection of uniform field 

The idea of applying the reassignment with N 
vectors on sequences with multidirectional movements 
such .as a zoom, for example, is not relevant. This is 
because, in this fairly specific case, the fact of 

30 adopting only N predominant vectors does not make it 
possible conveniently to process the fields consisting 
of multiple vectors. 

Figure 5 represents the image of the vectors 
during the zoom. It can easily be seen that the dispar- 

35 ity in the field does not allow any such uniformity. 

It is therefore decided to detect, in the first 
place, a field in which the vectors are uniformly dis- 
tributed, either unilaterally, or in all directions 
(zoom). This detection is conveyed by a standard devia- 
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tion of the first predominant vector close to the 
average standard deviation calculated from among the N 
predominant vectors. This is expressed as: 

if al < threshold*aaverage => uniform field present 
5 in which the threshold is fixed by the user 

(threshold = 1.34 for example). 

Examples relating to the types of movements which 
are successfully detected are represented in Figures 
6a, b, c, d. 

10 The objective is, at present, not to apply the 

algorithm when cases (c) and (d) are present. These 
cases have still to be distinguished from cases (a) and 
(b) . To do that the average values of the dx and dy 
movements are examined, from among the N adopted, and 

15 it is seen whether they are close to zero. This is be- 
cause it may be observed that the movements in a zoom 
seem to cancel out if they are added, in contrast to 
unilateral movement. A maximum difference of five pix- 
els can be set for dx, dy. 

20 Limitation on the temporal filtering 

It is useful not to have to filter the histo- 
grams temporally in the event of breaks in movement. It 
is possible: 

- to store the histogram of initial or reas- 
25 signed vectors for a P-type image; 

- at the next P-type image, P (t), the new "im- 
age" vectors are compared. If they differ too much from 
their counterparts arising from P (t - n), the original 
vectors are kept. 

30 Choice of the Number of Predominant Vectors 

The number of vectors necessary may be decided 
automatically and dynamically, in such a way that, for 
sequences with random movements (for example a sporting 
sequence) , there are more vectors than for sequences 

35 with uniform movements ("train"). 
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Claims 

1. Method of movement estimation including segmen- 

tation of the video image into image blocks, movement 
5 estimation per image block in order to obtain a move- 
ment vector field, characterized in that it includes a 
stage of reassignment of a vector to a block by select- 
ing one movement vector from among N predominant vec- 
tors belonging to the vector field. 
10 2. Method according to Claim 1, characterized in 

that, for a predominant vector,* second-order regional 
maxima are detected so as not to be taken into account 
during the selection of the other predominant vectors. 

3. Method according to Claim 1, characterized in 
15 that the predominant vectors are selected in each of 

the four directions. 

4. Method according to Claim 1, characterized in 
that the selection of the reassigned vector is based on 
the value of the inter-displaced-image difference 

20 (DFD) . 

5. Method according to Claim 4, characterized in 
that, if the DFDs associated with the N predominant 
vectors are greater than the DFD associated with the 
original vector, the zero vector is adopted. 

25 6. Method according to Claim 4, characterized in 

that, if the DFDs associated with the N predominant 
vectors are greater than the weighted DFD associated 
with the original vector, the original vector is kept. 

7. Method according to Claim 1, characterized in 
30 that the selection of the reassigned vector is based on 

the calculation of the activity (spatial gradient) in 
the inter-image difference block (current block - 
estimated block) . 

8. Method according to Claim 7, characterized in 
35 that, if the activities corresponding to the N predomi- 
nant vectors are greater than the activity correspond- 
ing to the original vector, the zero vector is adopted. 

9. Method according to Claim 7, characterized in 
that, if the activities corresponding to the N predomi- 
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nant vectors are greater than the weighted activity 
corresponding to the original vector, the original vec- 
tor is kept. 

10. Method according to Claim 4, characterized in 
5 that the components of the vectors used during the DFD 

calculations are the spatially filtered components. 

11. Method according to Claim 7, characterized in 
that the components of the vectors used during the spa- 
tial-gradient calculations are the spatially filtered 

10 components . 

12. Method according to Claim 1, characterized in 
that, for each image, the predominant vectors are cho- 
sen from among the field of vectors of the current 
image and the field of vectors of at least one preced- 

15 ing image. 

13. Method according to Claim 12, characterized in 
that the vectors of the preceding images, in addition 
to being scaled, are weighted as a function of the tem- 
poral distance. 

20 14. Method according to Claim 12, characterized in 

that, when a break in movement is detected, the vectors 
of the preceding images are not considered. 
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